Skip to content

Test dose trends and candidate shapes, and find lowest effective doses - #83

Merged
parashardhapola merged 7 commits into
masterfrom
dose/08-trend-contrasts-and-lowest-effective-dose
Nov 20, 2026
Merged

parashardhapola merged 7 commits into
masterfrom
dose/08-trend-contrasts-and-lowest-effective-dose

Conversation

@parashardhapola

Copy link
Copy Markdown
Member

The problem

dose_response (#81) says which genes respond to a dose in each cell type, through a spline F-test of one moderated fit, but not which doses differ from the vehicle, whether a response grows in dose order, which expected shape it follows, or from which dose it starts. Clinical pharmacology and toxicology ask exactly these questions. ICH E4 asks that the lowest dose recommended for use itself show an effect (ICH 1994); of 96 phase 2 dose-ranging trials in NEJM, The Lancet and JAMA (2018 to 2026) whose primary analysis could be determined, 66 tested each dose against placebo, 39 of them with a multiplicity procedure (a fixed sequence or hierarchy in 13, graphical or gatekeeping procedures in 7, Holm in 4, Dunnett in 3, Hochberg in 3), and MCP-Mod, which EMA qualified in 2014, was the primary analysis of 15. MCP-Mod's test step turns a few candidate shapes into the contrasts that best detect them and tests the largest of their t-statistics against the multivariate t (Bretz, Pinheiro and Branson 2005; Pinheiro et al. 2014). Toxicology reads each endpoint's lowest observed effect level from tests of each dose against the vehicle. In a dose series of 20,000 genes, calling each dose of every gene at 5% multiplies the false calls by the doses, so a gene's doses need an error rate over the genes as well as within the gene.

The solution

Three multiple contrast tests of each cell type's one fit, which dose_response runs when named, and each gene's lowest effective dose:

import cytepool as cp

result = cp.dose_response(
    cube, cp.Design(cube.samples, "~ 1"), dose=axis,
    tests=("spline", "dunnett", "williams", "mcp"), rng=1,
)
result.table("hepatocyte", "mcp")       # each gene's p-value, and the candidate whose contrast matches best
result.candidates                       # the shapes, spread over the design's doses
lowest = result.lowest_effective_doses("hepatocyte")  # the spline as screen, alpha 0.05
  hepatocyte: 9 dose groups, 27 samples, 817 genes; calls at 5% FDR: spline 88,
              dunnett 69, williams 73, mcp 92
  • "dunnett", "williams" and "mcp". Each dose against the vehicle (Dunnett 1955); the Williams-type test, the vehicle against the mean of the top one, two or more doses weighted by the cell type's samples (Bretz 2006); and the MCP step of MCP-Mod (multiple comparison procedure and modelling), the optimal contrast of each candidate shape for the cell type's doses and samples. Each is two-sided and refers the gene's largest moderated |t| to the multivariate t of Test which doses differ from the vehicle with multiple contrast tests #82, on the gene's df (under dream the smallest of its doses', an approximation), with the correlation that the gene's own covariance of its doses' estimates gives, voom's weights included; limma's approximate standard errors, which give no such covariance, are refused with them, as Test which doses differ from the vehicle with multiple contrast tests #82 refuses them. Its p_value is adjusted for the gene's contrasts (single-step), and adjusted_p_value and adjusted_p_value_across_groups adjust over genes by Benjamini-Hochberg. One recorded seed reproduces every p-value, whatever the other cell types and tests: each cell type and test draws from default_rng([seed, crc32(name), position]). For the MCP step, a gene's contrast names the candidate whose contrast best matches its changes, not a fitted shape.
  • cp.CandidateShapes. Emax, sigmoid Emax (the Hill equation), exponential and beta (an umbrella) shapes in DoseFinding's families. CandidateShapes.spread places two Emax shapes, a sigmoid Emax shape of Hill slope 3, an exponential shape and a beta shape over a series' doses along the dose axis; means gives each shape at any doses as DoseFinding's getResp does. A shape that does not change at a cell type's doses is left out there with the reason, and the printout says when more shapes than treated doses span fewer dimensions.
  • DoseResponseResult.lowest_effective_doses(group, screen=, alpha=). Stage-wise calls as stageR makes them (Heller et al. 2009; Van den Berge et al. 2017): the screen, the spline by default, calls R of G genes by Benjamini-Hochberg; in each, the doses' moderated t-tests are called by Holm's step-down procedure with Shaffer's first step at the level R alpha / G; the lowest dose called is the lowest effective dose. The calls control the overall false discovery rate (OFDR) of the genes, the expected share of the genes called with a false call; the share of false gene-dose calls is not controlled, and the calibration reports it.
  • Building blocks in cytepool.stats.multiple_contrasts: optimal_contrasts (DoseFinding's optContr) and stage_wise_adjusted_p_values (stageR's stageWiseAdjustment, "holm" or "none").

The Dose response chapter gains "Trends, doses against the vehicle and candidate shapes" and "From which dose a gene responds"; Multiple testing gains "Calling contrasts across genes"; Choosing a method two questions; the glossary seven entries; the bibliography six works; the API reference a page for CandidateShapes. The guide "Dose response per cell type" plots the spread shapes, tests them and finds each gene's lowest effective dose against the truth: on its hepatocytes the MCP step finds 223 of the 378 responses and calls 12 of the 1,491 flat genes, the spline 222 and 8, and the Williams-type test 165 and 7; 165 of the 230 genes the spline calls get a lowest effective dose, every one above the gene's true 1.1-fold benchmark dose (median 7.3 times). The chapter shows how to get each dose's own p-value, adjusted for the gene's doses, from multiple_contrast_test on the cell type's fit.

Why it matters

The first question of a dose series, which genes respond in each cell type (#81), now has its follow-ups on the same fit: which doses differ, whether the response grows in dose order or follows an expected shape, and from which dose a gene responds, with an error rate over the genes. It builds on #82 (the multivariate t, multiple_contrast_test, Dunnett's and Williams-type contrasts) and #81 (dose_response and its private dose-group estimates, which gain a dream branch). PR 9's gate can take these tests and default screen to its gate; PR 13 fits the candidate shapes, MCP-Mod's modelling step; PR 19 adds one-sided tests; PRs 21 and 22 run the MCP step on published series.

Validation

Each script ran twice in one Modal container (4 CPU, 8 GiB, AMD Zen 3) and printed the same rows both times. validation/dose_contrasts.py, run again at the final code after code review retired its jobs of limma's standard errors, printed 80 checks, 71 pass, 7 deliberate differences and 2 information, 0 failed; the second run took 3 min 23 s with a peak memory of 940 MiB. Each reference runs as a program of its own on numbers only: DoseFinding 1.4-2, multcomp 1.4-32, mvtnorm 1.4-2 and stageR 1.28.0; limma 3.62.1 and edgeR 4.4.0 fitting each cell type from its counts in R, and variancePartition 1.36.2 for dream; scipy 1.18.1 and mpmath 1.3.0 in a process without CytePool.

Check family Reference Largest difference Tolerance Outcome
CandidateShapes.spread: ED50s, Hill slopes, beta parameters and scale, 60 cases on 5 ladders the rule's positions in R; DoseFinding guesst 1.0e-13 1e-10 5 pass
The exponential shapes' delta, 60 cases mpmath at 40 digits; guesst, a root search 6.2e-16; 2.0e-6 1e-10; 1e-5 2 pass
CandidateShapes.means, 6 sets of shapes at 3 to 9 doses, beta peaks between doses and above the top DoseFinding getResp(Mods(...)) 3.8e-11 1e-10 6 pass
optimal_contrasts: groups of sizes n, a general covariance, the vehicle's row dropped; 6 designs of 3 to 9 groups DoseFinding optContr 1.6e-13 1e-10 3 pass
The MCP step on seeded estimates at 5, 12 and 40 df (144 features): contrasts and t MCTtest; multcomp glht(parm()) 4.1e-13 1e-10 2 pass
Its p-values from 0.001 to 1: 303 of full rank, 357 of singular correlations multcomp summary(glht()), mvtnorm, error over 3 2.75 and 3.82 combined SE 5.5 2 pass
stage_wise_adjusted_p_values, "none" and "holm", alpha 0.05 and 0.1, 1 to 8 hypotheses, ties and missing values stageR stageWiseAdjustment 2.5e-16 1e-10 4 pass
Every call of "holm" (2,420) the closure of Bonferroni tests, enumerated in R identical identical 1 pass
dose_response: genes, doses compared and samples per dose, 15 cell types R from the counts and cells identical identical 1 pass
Dunnett's, Williams-type and MCP tests: contrasts, each gene's largest |t|, its contrast and the df; limma-voom and limma-trend (3,170 genes each) limma from the counts, then glht(parm()) on contrMat and optContr 3.0e-13 1e-10 24 pass
The same under dream (239 genes), the multivariate t on the smallest df of a gene's doses variancePartition dream, then multcomp 1.6e-8; df 8.2e-8 1e-6; df 1e-5 12 pass
The three tests' p-values at non-integer df: 800, 800 and 300 genes from 0.001 to 1 scipy multivariate_t.cdf, 8 calls, on R's t, correlations and df 3.38 combined SE 5.5 3 pass
lowest_effective_doses, spline and dose-group screens: doses called, the screen's calls, the adjusted p-value at the lowest effective dose stageR on the screen's and each dose's p-values 5.2e-12 1e-10 4 pass
Each gene's lowest effective dose, 6,579 and 6,340 genes the closure of Bonferroni tests identical identical 2 pass
The MCP step's p-values below 1e-3 (12), and those of singular cell types (120), which lie between multcomp's at the integer df below and above the gene's multcomp listed n/a 2 information
An exponential shape of delta D / 1000, where DoseFinding's exp(d / delta) - 1 overflows to NaN at the top dose; MCTtest's |t| for two-sided tests; df of 12.7, which MCTtest and summary(glht()) refuse; one hypothesis per feature, where stageR's "holm" fails, at alpha 0.05 and 0.1; a called feature with one hypothesis left of four, to which stageR gives 0 and CytePool Bonferroni at the screen's level DoseFinding, multcomp, stageR n/a 7 deliberate differences

R warned 244 times, every warning mvtnorm's "Completion with error > abseps" (171 in summary(glht()), 73 in MCTtest), on p-values whose reported error exceeds mvtnorm's default absolute error of 1e-3; the comparisons use each p-value's reported error, and the 12 p-values below 1e-3 are not compared.

validation/dose_response_calibration.py printed 101 checks, 97 information and 4 controls, 0 failed, and #81's 53 rows as recorded; the second run took 41 min 52 s with a peak memory of 447 MiB. The three tests ran on the first 200 null datasets and the 10 power datasets of each design cell of the core grid (4 and 9 dose groups, 2 and 3 replicates, 2 cell types of 1,000 genes), and the stage-wise calls of lowest_effective_doses, with the spline as screen, on the power datasets:

Design Test Null: p below 0.05 Null: cell types with a call at 5% FDR Power at 5% FDR False discovery proportion
K = 4, 2 replicates Dunnett's 0.051 0.142 0.225 0.050
K = 4, 2 replicates Williams-type 0.052 0.135 0.236 0.050
K = 4, 2 replicates MCP step 0.052 0.170 0.272 0.051
K = 4, 3 replicates Dunnett's 0.050 0.052 0.330 0.030
K = 4, 3 replicates Williams-type 0.051 0.058 0.344 0.035
K = 4, 3 replicates MCP step 0.051 0.075 0.377 0.024
K = 9, 2 replicates Dunnett's 0.051 0.110 0.232 0.050
K = 9, 2 replicates Williams-type 0.052 0.113 0.267 0.057
K = 9, 2 replicates MCP step 0.053 0.083 0.421 0.062
K = 9, 3 replicates Dunnett's 0.050 0.050 0.329 0.040
K = 9, 3 replicates Williams-type 0.051 0.055 0.353 0.034
K = 9, 3 replicates MCP step 0.052 0.045 0.489 0.050
Design Overall false discovery proportion False gene-dose calls Genes the spline calls with a dose called Lowest effective dose over the true 1.1-fold BMD: median (quartiles)
K = 4, 2 replicates 0.056 (SE 0.011) 33 of 836 582 of 681 14 (5.0 and 60)
K = 4, 3 replicates 0.033 (SE 0.006) 21 of 1,112 771 of 885 14 (5.5 and 51)
K = 9, 2 replicates 0.057 (SE 0.006) 38 of 1,531 653 of 1,075 11 (4.9 and 26)
K = 9, 3 replicates 0.048 (SE 0.006) 18 of 2,246 869 of 1,245 8.9 (4.0 and 21)

Under the null the shares of p-values below 0.05 were 0.050 to 0.053 (standard error 0.0005), the MCP step's the highest, as the spline's are, and a cell type of 1,000 null genes had a call in 8.3% to 17.0% of the datasets with 2 replicates and 4.5% to 7.5% with 3: the tail that moderation at few residual degrees of freedom gives #81's F-tests too. The MCP step found as many responsive genes as the spline at 4 dose groups (0.272 and 0.377, the spline 0.265 and 0.371, standard errors about 0.01) and more at 9 (0.421 and 0.489 against 0.386 and 0.466), and as many bells as the spline at 9 dose groups (0.44 and 0.51, the spline 0.44 and 0.53, the Williams-type test 0.17 and 0.26). The stage-wise calls kept the overall false discovery proportion within 1.2 standard errors of 0.05, with 0.8% to 3.9% of the gene-dose calls false, and the lowest effective dose lay a median of 8.9 to 14 times above the gene's true benchmark dose of a 1.1-fold change. The four controls, calls of each dose at an unadjusted 0.05, called a dose in all 400 null cell types of each design cell, as they must.

Modal runs, each an ephemeral app through the reference harness:

Time What Why Resources Duration Result
10:07 Probes of DoseFinding, multcomp and variancePartition (pr08-build-probe) How Mods scales a beta shape whose peak lies above the doses, whether glht(parm()) takes singular correlations and non-integer df, where dream keeps each gene's covariance 2 CPU, 4 GiB 16 s in the container Beta scaled at its peak wherever it lies; glht refuses non-integer df only in summary; dream's cov.coefficients.list
10:14 to 10:22 The validation's three R programs, 5 runs (pr08-r-side to -5) Writing them 2 CPU, 4 GiB 36 to 67 s each Runs 1 to 4 stopped at errors in the programs (a data frame of cases, optContr with the placebo dose, DoseFinding's overflow at delta = D / 1000, stageR with one column); run 5 passed
10:56 dose_contrasts.py on an overlay of master, PR 4 (e0b9439), PR 7 (dfb50da) and this PR The first full run 4 CPU, 8 GiB 158 s Failed at a shadowed variable in the script
11:03 The same, fixed The first complete run 4 CPU, 8 GiB 357 s 95 checks, 15 failed: stageR's "holm" on inputs it cannot take, dream without variancePartition's span of 0.5, and the scipy rows of the dream job
11:10 stageR's "holm" with missing values (pr08-stager-na-probe) Which patterns stageR takes 2 CPU, 4 GiB 7 s One hypothesis left of three or more gets 0; fewer fail; the deliberate differences
11:20 dose_contrasts.py --write-fixtures on the overlay Fixtures for the tests 4 CPU, 8 GiB 338 s 92 checks, 0 failed
13:46 dose_contrasts.py --write-fixtures on PR 7's merged code (ff11d1f), PR 4's final code (b6ea399) and this PR's The design on the merged and final code of both PRs 4 CPU, 8 GiB 349 s 92 checks, 0 failed; the same rows as at 11:20
13:46 dose_contrasts.py --benchmark The cost of the three tests in dose_response 4 CPU, 8 GiB 364 s 0.48, 1.13 and 0.50 ms per gene at 7 doses; the costs under Decisions
13:57 dose_contrasts.py --write-fixtures twice in one container The recorded run and the fixtures 4 CPU, 8 GiB 561 s 92 checks, 0 failed, the same rows; superseded when the reference line was fixed
13:57 dose_response_calibration.py twice in one container The recorded run of the calibration's new rows 4 CPU, 8 GiB 20 min until Modal preempted the container, then 5,124 s in the container it restarted 101 checks, 0 failed, 97 information and 4 controls, the same rows in both runs, PR 7's 53 rows as recorded; second run 2,511.8 s, 447 MiB
14:06 dose_contrasts.py with an R profile that prints warnings (pr08-r-warnings) What R warned about 4 CPU, 8 GiB 412 s No listing: Rscript --vanilla skips the profile
14:13 dose_contrasts.py twice, after the reference line printed R once The recorded run 4 CPU, 8 GiB 336 s The same rows; on an Intel host with AVX-512, so its time was not recorded
14:17 dose_contrasts.py with each Rscript behind a wrapper that prints warnings (pr08-r-warnings-2) What R warned about 4 CPU, 8 GiB 317 s 244 warnings, all mvtnorm's "Completion with error > abseps"
14:20 dose_contrasts.py --write-fixtures twice in one container The recorded run 4 CPU, 8 GiB 644 s 92 checks, 0 failed, the same rows as every earlier run; second run 239.1 s, 956 MiB; fixtures identical to the committed ones
14:33 The CI documentation build (Node 24, Sphinx with warnings as errors, MyST strict) The docs workflow 4 CPU, 8 GiB 24 s Failed on 4 warnings: the API role on an API page, two method references in the module docstring, and two docstring lines that Napoleon read as types
14:35 The same, fixed The docs workflow 4 CPU, 8 GiB 62 s Passed
15:44 The CI documentation build at the final head The docs workflow 4 CPU, 8 GiB 42 s Passed
15:53, 15:56, 15:58 CI's check of the guide's saved outputs, make -C docs check-outputs PAGES="guides/dose_response", with the harness's AVX2 settings unset Whether a host with AVX-512, as GitHub's runners have, prints the guide's outputs as saved 4 CPU, 8 GiB twice; 8 CPU, 8 GiB 146 s, 135 s, 119 s Outputs current each time; the first two hosts were AMD Zen 3 without AVX-512, the third an Intel host with it
17:25 dose_contrasts.py --write-fixtures twice in one container, after code review The run without limma's standard errors, and the fixtures 4 CPU, 8 GiB 327 s 80 checks, 0 failed, the same rows in both runs; an Intel host with AVX-512, so its time was not recorded
17:32 The same The recorded run, on the Validation page's host 4 CPU, 8 GiB 548 s The same rows; AMD Zen 3; second run 203.2 s, 940 MiB; the same fixtures
17:43, 17:45 The CI documentation build after code review, and again after ruff formatted a code block and the guide's cells The docs workflow 4 CPU, 8 GiB 80 s, 64 s Passed both times
17:48 CI's check of the guide's saved outputs with the harness's AVX2 settings unset Whether a host as GitHub's runners prints the guide's new outputs as saved 8 CPU, 8 GiB 165 s Outputs current, on an AMD Zen 3 host without AVX-512
18:10 CI's check of the guide's saved outputs with the harness's AVX2 settings unset, then the CI documentation build, at the final head The docs workflow and the guide's outputs at the code that merges 8 CPU, 8 GiB 190 s Outputs current and the build passed, on an AMD Zen 3 host without AVX-512

docs/changed_since_run.py lists 34 scripts on master after #82, whose changes to cytepool/design/_contrast_weights.py and cytepool/stats/_result_tables.py reach most scripts, and 35 on this branch: those of master and multiple_contrast_tests.py. This PR records dose_contrasts.py. dose_response_calibration.py ran at 13:57; code review then changed code it checks, the scaling in optimal_contrasts and the sigmoid Emax shape in logarithms, which move the MCP step's contrasts by about one unit in the last place. Its numbers stand, so it was not run again: on 12 of its datasets (3 null and 2 power datasets of each design cell with 2 replicates), run through the calibration's own code at both heads, the three tests' p-values agreed within 1.1e-14 (relative) and their t-statistics within 2.4e-15, and the contrasts named, the lowest effective doses and their stage-wise adjusted p-values were identical, while the printed rows round to 3 or 4 digits. The PR's code reaches 14 others without changing what they check, so they were not run again: multiple_contrast_tests.py through cytepool/stats/multiple_contrasts.py (two functions added, and _check_symmetric takes the argument's name); dose_axes.py, dose_design.py, dose_response_tests.py and simulate_dose_series.py through cytepool/_doses.py, which takes DoseAxis's reading of a sample table's doses unchanged; and activity.py, combined_p_values_and_replication.py, gene_set_files_and_ora.py, gsea.py, pathway_scores.py, pca_and_dot_plots.py, plotting.py, sumrank.py and variance_partitioning.py through cytepool.plotting, which imports cytepool.dose_responses.

Tests

New tests: 19 in tests/test_candidate_shapes.py (35 with their cases), 12 in tests/test_multiple_contrasts.py (42) and 19 in tests/test_dose_responses.py (20), with the new callables in the masked-input and top-level registries. They check against independent references: DoseFinding 1.4-2, stageR 1.28.0, multcomp and R's limma fits through the fixtures that validation/dose_contrasts.py --write-fixtures wrote to tests/data/dose_contrasts (635 KB); values derived by hand (the optimal contrast of mu = (0, 1, 2) for n = (2, 1, 1), proportional to (-1.5, 0.25, 1.25); a stage-wise table worked by hand; the series expansion of a nearly linear exponential shape); a closed test of Bonferroni tests enumerated in the test; and invariants that a wrong implementation breaks: unit invariance of the shapes and contrasts, the non-centrality that the optimal contrast maximises, Dunnett's t equal to each dose's own t, dream's df as the smallest of a gene's doses, p-values independent of threads, cell-type order, other cell types and the other tests named, the seed rule pinned by a direct call of multiple_contrast_test, contrasts unchanged by shapes and covariances scaled to 1e-200, 1e200 and 1e-300, and steep sigmoid shapes far from their ED50 that overflow nowhere. A test runs the chapter's new examples, each dose's own p-value included, and compares their printouts and stated numbers with the simulator's truth.

Mutations, each run against the tests and each caught (test that caught it); two of them, Shaffer's first multiplier m and the lowest effective dose as the highest called, were run again at the final head, on a copy of the package, and caught again:

Mutation Caught by
Shaffer's first multiplier m instead of m - 1; G counting missing screens; the cap before scaling by G / R; no running maximum test_stage_wise_matches_stager
The screen at q < alpha instead of q <= alpha test_stage_wise_edge_cases
Contrasts centred on the unweighted mean; S instead of S^-1 test_optimal_contrasts_match_optcontr
A sigmoid slope of 1; the exponential's half change at 0.8 D; positions on the dose scale test_spread_matches_the_rule_and_guesst
The beta shape scaled at the doses' largest value instead of its peak test_means_match_dosefinding
The exponential without expm1 test_a_nearly_linear_exponential_keeps_its_digits
The lowest effective dose as the highest called; Holm on the adjusted per-dose p-values test_lowest_effective_doses_match_stager
Williams-type weights from equal group sizes; one-sided tests; a one-sided Williams-type test; the MCP step's vehicle row kept test_multiple_contrast_tests_match_r[voom]
dream's df as the largest of a gene's doses test_dunnett_t_is_each_dose_s_t
The seed from Python's hash of the cell type test_contrast_tests_do_not_depend_on_threads_order_or_other_cell_types
Flat shapes kept test_a_flat_shape_is_left_out
The change at the top dose, not at the lowest effective dose test_lowest_effective_doses_by_the_closed_test
The seed's position counted among the tests named test_contrast_tests_do_not_depend_on_threads_order_or_other_cell_types
adjusted_p_value of the three tests left unadjusted test_the_contrast_tests_read_back
A beta_scale equal to the top dose refused test_a_beta_scale_at_the_top_dose_is_accepted
The dose-group test run with one treated dose when only multiple contrast tests are named test_few_dose_groups
A shape flat only when exactly 0; the printout without the shapes left out test_a_flat_shape_is_left_out
The symbol dropped from the lowest effective doses test_the_contrast_tables_carry_the_genes_symbols
n_threads not passed on test_the_contrast_tests_follow_the_thread_limit
limma's approximate standard errors accepted test_contrast_test_arguments_are_refused_with_a_fix
Optimal contrasts of the shapes and covariance unscaled test_optimal_contrasts_do_not_depend_on_the_scale, test_a_flat_shape_is_left_out
The sigmoid Emax shape as a ratio of powers test_a_steep_sigmoid_far_from_its_ed50_neither_overflows_nor_warns
Booleans accepted as shapes test_optimal_contrasts_refuse

Coverage at the final head (coverage 7.13.1, branches; none of the new code is compiled by numba, so the pass ran with the JIT on), over the tests of the candidate shapes, multiple contrasts, dose responses, dose axes, dose designs, masked inputs and the top level: cytepool/candidate_shapes.py, cytepool/dose_responses.py, cytepool/stats/multiple_contrasts.py, cytepool/_doses.py and cytepool/dose_axes.py 100% of 1,393 statements and 442 branches (1,995 tests).

Checks at the final head: the full suite 6,023 passed and 18 skipped (zstd needs Python 3.14); ruff check and ruff format --check on cytepool tests validation docs clean; mypy cytepool clean (94 files); the Sphinx API reference built here with warnings as errors, and the whole CI documentation build (Node 24, Sphinx with warnings as errors, MyST strict) passed on Modal, since this machine has no Node.js; make -C docs check-outputs PAGES="guides/dose_response" found the guide's outputs current, here and on Modal with the harness's AVX2 settings unset (before code review on an Intel host with AVX-512, as on GitHub's runners; after it on AMD hosts without it). The guide's two statements that no cell prints, that every lowest effective dose lies above the gene's true 1.1-fold benchmark dose and that two of the 165 genes with a dose called are flat, were checked by running its code: the smallest ratio was 1.07, and the two genes without a finite benchmark dose are the two flat ones.

Decisions

  • Holm's step-down with Shaffer's first step within each gene, as stageR does, rather than a step-down Dunnett procedure or a fixed sequence from the top dose. It is valid for any shape, the screen serving as the closure's global test, exact against stageR and an enumerated closure, and cheap; a fixed sequence stops at the top dose of a bell. The correlation through the shared vehicle that it leaves out is worth 22% of a p-value for 3 doses and 65% for 8 at 5 df, and 1% and 3% at 40 df, at single-step p-values of 1e-6 (Test which doses differ from the vehicle with multiple contrast tests #82's table).
  • One hypothesis left in a called gene takes the multiplier 1, Bonferroni at R alpha / G, which holds whatever the screen tested; stageR's "holm" gives it 0, or stops when the gene had two hypotheses or one (deliberate differences).
  • Missing hypotheses follow stageR: they leave their gene's family, and Shaffer's first step then needs the screen to have tested the gene's other hypotheses only. The docstring and the chapter say so, with the remedy for a screen that tested more (Holm within each gene, then method="none", which held 0.050 at 0.05 where Shaffer's step gave 0.073 in a simulation); Holm's m in rows with a missing value would have parted from stageR on every such row. dose_response passes no missing p-values.
  • lowest_effective_doses computed on demand with screen= and alpha=: stage-wise values hold only at the level given, so no level is hidden in a table, and PR 9's gate can supply the screen.
  • The spline as the default screen, with no fallback: where a cell type did not run the spline, lowest_effective_doses raises a KeyError that says why and names the tests of any response it ran, rather than taking the next test silently, so one call never rests on different screens in different cell types. "linear_trend" and "nonlinearity" are refused as screens, since they test a line or a departure from one, not any response.
  • Names. CandidateShapes rather than Candidates, which does not say of what at the package's top level; each field names a DoseFinding model family, so beta rather than umbrella, with "umbrella shape" in its docstring and the glossary; spread's counts are n_emax, n_sigmoid_emax, n_exponential and n_beta, since emax=2 is an ED50 in DoseFinding's Mods; the level is alpha, as stageR and CytePool's other calls name it; the column is stage_wise_adjusted_p_value, since adjusted_p_value means Benjamini-Hochberg within the cell type in every other table. Choosing a method asks "Does a response follow one of a few expected shapes?" rather than "Is there a dose signal under shape uncertainty?", in the words of the chapter's table.
  • One set of contrasts per cell type, from its doses and samples, tested with each gene's own covariance, as Test which genes respond to a dose in each cell type #81 weighs the spline; per-gene optimal contrasts were trimmed.
  • The three tests run only when named, for their cost: 0.48, 1.13 and 0.50 ms per gene for Dunnett's, the Williams-type and the MCP tests of 7 doses, against 0.87 s for the default tests of 20,000 genes. The Williams-type test's 1.13 ms per gene is above a target of 1 ms per gene and within 1.5 times it, so its cost was left as it is (Test which doses differ from the vehicle with multiple contrast tests #82 measured 1.13 to 1.67 ms one-sided).
  • Two-sided tests, as genes rise or fall; one-sided tests are PR 19's.
  • The beta model as the umbrella, scaled to 1 at its peak wherever the peak lies, as DoseFinding's Mods does; spread places shapes along the axis's positions, the exponential on the dose scale, with a sigmoid Emax slope of 3.
  • Dream supported, the multivariate t on the smallest df of a gene's doses, an approximation that the chapter states with its size: conservative for Dunnett's test and, in a simulated unbalanced design of donors, for the Williams-type test, while the MCP step's contrasts had fewer df in 19 to 174 of 240 genes, down to 0.83 times. The tests' own df would need their contrasts in dream's fit, left for a later PR.
  • limma's approximate standard errors refused with the three tests (contrast_standard_errors="limma"), as Test which doses differ from the vehicle with multiple contrast tests #82's multiple_contrast_test refuses such fits: they give no covariance of a gene's doses' estimates, and with them the Williams-type and MCP p-values of a simulated 9-group series moved by up to 100-fold.
  • optimal_contrasts takes named shapes only, a mapping as DoseFinding's optContr takes named models; it scales the shapes and the covariance before solving, so shapes of any magnitude give their contrasts.
  • One recorded seed: each cell type and test draws from its own generator, so a cell type's p-values do not depend on the other cell types or the threads.
  • Trimmed: per-contrast tables of the three tests, out (about 6 MB per test and cell type of 20,000 genes; each dose's calls come from lowest_effective_doses, and PR 13's modelling step recomputes the shapes it fits); critical values and constrained contrasts (MCTtest(critV), optContr(type = "constrained")), out, as p-values do not need them; DoseFinding's quadratic umbrella, to PR 13's model set if fitting needs it; MCP-Mod's modelling step, PR 13.
  • Test which doses differ from the vehicle with multiple contrast tests #82's _check_symmetric takes the argument's name, "covariances" by default, so that optimal_contrasts names covariance; Test which doses differ from the vehicle with multiple contrast tests #82's own messages are unchanged.

The MCP step of MCP-Mod tests a dose signal against a few expected shapes of response, each turned into the contrast of the dose groups that best detects it, and calling each dose of thousands of genes needs an error rate over the genes, not only within each gene. CytePool had neither the shapes nor these two procedures.

CandidateShapes holds Emax, sigmoid Emax, exponential and beta shapes in DoseFinding's model families, gives each shape's means at any doses as DoseFinding's getResp does, and places shapes over a series' doses with spread: positions along the dose axis between the lowest and highest treated dose, an exponential shape that reaches half its change late, and a beta umbrella that peaks mid-range. optimal_contrasts builds each shape's contrast, proportional to S^-1 (mu - m 1), as optContr does. stage_wise_adjusted_p_values gives stageR's stage-wise adjusted p-values: a Benjamini-Hochberg screen calls R of G genes, and in each gene called Holm's step-down procedure with Shaffer's first step runs at the level R alpha / G, which controls the overall false discovery rate of the genes. DoseAxis and CandidateShapes read a sample table's doses through one shared function.

The tests compare the shapes, contrasts and adjusted p-values with DoseFinding 1.4-2 and stageR 1.28.0 fixtures, values derived by hand, a closed test of Bonferroni tests enumerated in the test, and invariants such as unit invariance and the non-centrality the optimal contrast maximises.
A gene's spline test says that it responds to a dose, not which doses differ from the vehicle, whether the response grows in dose order, which expected shape it follows, or from which dose it starts. Calling each dose against the vehicle in every gene multiplies the false calls by the doses.

dose_response runs three multiple contrast tests of each cell type's one moderated fit when they are named in tests: "dunnett", each dose against the vehicle; "williams", the Williams-type test of the vehicle against the pooled top doses, weighted by the cell type's samples; and "mcp", the MCP step of MCP-Mod on candidate shapes, given or spread over the design's doses. Each is two-sided, takes the gene's own covariance of its doses' estimates, voom's weights included, and the smallest df of its doses under dream, and adjusts across genes by Benjamini-Hochberg within and across cell types. One recorded seed reproduces the p-values, each cell type and test drawing from it, its name and its position. DoseResponseResult.lowest_effective_doses calls each dose in the genes that a test of any response calls, the spline by default, as stageR does, and gives each gene's lowest effective dose.

The tests check the contrasts, statistics and df against limma and multcomp fixtures, the p-values against scipy within simulation error, the lowest effective doses against stageR and a closed test, Dunnett's t-statistics against each dose's own, and the refusals, records and seeds.
validation/dose_contrasts.py checks the candidate shapes against DoseFinding's guesst, getResp and the rule computed in R, with mpmath for the exponential shapes' roots; optimal_contrasts against optContr; the MCP step on seeded inputs against MCTtest and multcomp's glht, singular correlations included; the three tests of dose_response against limma, edgeR and variancePartition fits made in R from the counts, with multcomp's contrMat and optContr; their p-values at non-integer df against scipy; and the stage-wise calls and lowest effective doses against stageR and a closed test enumerated in R. It writes the fixtures in tests/data/dose_contrasts and, with --benchmark, times dose_response with each test on 20,000 genes.

validation/dose_response_calibration.py gains the three tests' null and power rows on the core grid, with a control row of per-dose calls at an unadjusted 0.05, and the stage-wise calls of lowest_effective_doses: their overall false discovery proportion, the false gene-dose calls, the share of screened genes with a dose called, and the lowest effective dose against the true benchmark dose.
The Dose response chapter gains "Trends, doses against the vehicle and candidate shapes", on the three multiple contrast tests, the candidate shapes and their optimal contrasts, with the costs measured, and "From which dose a gene responds", on the stage-wise calls, what the lowest effective dose is and is not, and every dose called; with defaults, limits and checks. Multiple testing gains "Calling contrasts across genes", on stageR's procedure and the overall false discovery rate it controls, linked from the reading of a gene's contrasts at alpha R / G. Choosing a method gains two questions, the glossary seven entries, the bibliography Bretz et al. (2005), Pinheiro et al. (2014), Duda et al. (2022), ICH E4 (1994), Shaffer (1986) and Heller et al. (2009), and the API reference a page for CandidateShapes.

The guide "Dose response per cell type" tests a few expected shapes on its simulated series and finds each gene's lowest effective dose, against the truth. A test runs the chapter's new examples and checks their printouts and stated numbers against the simulator's truth.
The Validation page gains a section for validation/dose_contrasts.py, with what its recorded run printed, its time and memory, a row in the summary, the benchmark's times and the commands that reproduce it. The section on the error rates and power of the dose tests gains the rows of the three multiple contrast tests, their null shares, calls, power and false discovery proportions, and those of the stage-wise calls of lowest_effective_doses: their overall false discovery proportion, the false gene-dose calls, the genes with a dose called and the lowest effective dose against the true benchmark dose. The Dose response chapter cites the power and the overall false discovery proportion they found.

docs/validation_runs.toml records the digests of the code that the two runs checked.
Optimal contrasts of shapes whose changes were tiny or huge underflowed or overflowed to infinite or zero weights, and a steep sigmoid Emax shape far from its ED50 raised OverflowError, so dose_response could fail in a cell type that keeps only low doses. With contrast_standard_errors="limma", the three multiple contrast tests took each gene's covariance from limma's approximation, which multiple_contrast_test refuses and which moved Williams-type and MCP p-values by up to 100-fold. The chapter said that "dunnett" gives each dose's own result, did not say that Shaffer's first step needs the screen to have tested the hypotheses given, and called dream's smallest dose df conservative for every test.

optimal_contrasts now scales the shapes and the covariance before it solves and takes named shapes only; the sigmoid Emax shape is computed in logarithms; dose_response refuses the three tests with limma's approximate standard errors, and its printout names the shapes left out. The chapter shows each dose's own adjusted p-value through multiple_contrast_test, what the MCP step's contrast names, EMA's minimum by step, and dream's df as an approximation with its size; the guide plots the candidate shapes, counts flat calls and shows the contrast each called gene names. Private helpers lose a defensive guard, a second reader of numbers and a second array form of shapes.

validation/dose_contrasts.py ran again without the jobs of limma's standard errors: 80 checks, 0 failed. New tests pin the seed of a test named alone, the change at each lowest effective dose, the adjustment within a cell type, symbols, the thread limit, a beta scale at the top dose and shapes at extreme scales; 15 deliberate bugs were each caught.
The ValueError entry of dose_response's docstring broke "the design names" onto a line of its own, which help() shows as a stray short line. The entry now wraps as the rest of the docstring does; its text is unchanged.
@parashardhapola
parashardhapola merged commit 9fa2cc1 into master Nov 20, 2026
6 checks passed
@parashardhapola
parashardhapola deleted the dose/08-trend-contrasts-and-lowest-effective-dose branch November 20, 2026 17:36
parashardhapola added a commit that referenced this pull request Nov 21, 2026
A dose series runs several tests of each gene, and nothing said which of them decides that a gene responds, at which false discovery rate, or how a fold change enters that decision. Toxicology's prefilters cut the estimated change after the test, and in vehicle-only data or under permuted dose labels they found responsive genes in most runs (Lee et al. 2024; Bruns et al. 2026); at 2 replicates per dose, a cell type of 1,000 null genes had a spline call in 10% to 16% of simulated series.

dose_response takes gate=, one recorded test of any response at a stated FDR after the dose-blind expression filter: the spline by default, the dose groups, Dunnett's, the Williams-type or the MCP test, or a fold change that TREAT tests at each dose, with Bonferroni over the doses and Benjamini-Hochberg across genes. Each cell type gains the table "gate" with passes_gate, and lowest_effective_doses reads the gate in place of the screen and alpha that #83 added. permutation_check runs the gate again on dose labels permuted among the samples that the design treats alike, every assignment where they are few, and gives a permutation p-value for whether the true doses pass more genes than permuted ones, with the smallest p-value the runs can give.

Each result now records which genes count as responding and how that was decided, and the check answers at the user's own design whether those calls could come from noise. validation/dose_gate.py matches limma, edgeR, stageR and variancePartition fitted by R from the counts, and R's own enumeration and refits of the assignments, call for call; on simulated null series the default gate stays within the bound that 2,000 cell types allow at 3 replicates and exceeds it at 2, a known limit, and the check's runs pass genes as often as null series do.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant